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Probabilistic w-automata are variants of nondeterministic automata for infinite words where all choi- 
ces are resolved by probabilistic distributions. Acceptance of an infinite input word can be defined 
in different ways: by requiring that (i) the probability for the accepting runs is positive (probable 
semantics), or (ii) almost all runs are accepting (almost-sure semantics), or (iii) the probability mea- 
sure of the accepting runs is greater than a certain threshold (threshold semantics). The underlying 
notion of an accepting run can be defined as for standard w-automata by means of a Biichi condition 
or other acceptance conditions, e. g., Rabin or Streett conditions. In this paper, we put the main fo- 
cus on the probable semantics and provide a summary of the fundamental properties of probabilistic 
^-automata concerning expressiveness, efficiency, and decision problems. 

1 Introduction 

While classical finite automata can serve to recognize languages over finite discrete structures, u;-au- 
tomata are acceptors for languages consisting of infinite objects. They have been applied in various 
research areas, including the verification of reactive systems and reasoning about infinite games and 
decision problems for certain logics. Many variants of cj-automata have been studied in the literature 
that can be classified according to their inputs (e. g., words or trees), their acceptance conditions (e. g., 
Biichi, Rabin, Streett, Muller or parity acceptance) and their branching structure (e. g., deterministic, 
nondeterministic, or alternating). We refer to |fT9H 71 for an overview of automata over infinite objects. 

Probabilistic variants of cj-automata for languages over infinite words have been recently introduced. 
Their syntax is roughly the same as for probabilistic finite automata (PFA) |[T5l[T4ll . i. e., they are finite- 
state automaton where for each state q and input letter a probability distribution specifies the probabilities 
for the sucessor states. Furthermore, they are equipped with an acceptance condition as in nondetermin- 
istic w-automata. The accepted language of a probabilistic w-automata can be defined by imposing a 
condition on the acceptance probability for the input words. Under the probable semantics, acceptance 
of an infinite word a = a\ a-i CI3 . . . requires that the generated sample run for a (i. e., sequence of states 
that are passed in the automaton while reading a letter by letter) meets the acceptance condition with pos- 
itive probability. The probable semantics is in the spirit of nondeterministic automata where the accepted 
words are those words that have at least one accepting run. The almost-sure semantics of a probabilistic 
iij-automata can be understood as the probabilistic counterpart to universal automata as it requires that the 
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accepting runs have probability measure 1, i. e., almost all runs are accepting. The threshold semantics 
follows the concept of PFA and deals with a fixed threshold A e]0, 1 [ and classifies an input word a to be 
accepted if the probability of the accepting runs for a is greater than A. 

The different semantics yield different classes of recognizable languages over infinite words. Most 
powerful is the threshold semantics which covers the class of w-regular languages, but also non-u;- 
regular languages. Given the results for PFA which are known to be more expressive than standard 
finite automata, this is not surprising. While PFA with the probable semantics agree with ordinary non- 
deterministic automata, probabilistic automata with Biichi or other standard acceptance conditions and 
the probable semantics are strictly more expressive than their nondeterministic counterparts. Further- 
more, there are languages L n that are recognizable by probabilistic Biichi automata of linear size, while 
smallest nondeterministic cj-automata for L n have exponentially many states. For nondeterministic oj- 
automata it is well-known that Biichi acceptance is as powerful as, e. g., Streett or Rabin acceptance, 
but the transformations from nondeterministic Streett automata to nondeterministic Biichi automata can 
cause an exponential blow-up lfl7l . In contrast, there is a polynomial transformation from probabilistic 
Biichi to probabilistic Streett automata, both under the probable semantics. Concerning the standard 
composition operators (union, intersection and complementation), the class of languages that are rec- 
ognizable by probabilistic w-automata under the probable semantics enjoys the same properties as the 
class of w-regular languages. Both are closed under all three operators. Union and intersection can eas- 
ily be realized by means of sum and product constructions, respectively. Complementation, however, is 
"difficult" and relies on a complex powerset construction that can cause an exponential blow-up. The 
price we have to pay for the extra power of probabilistic o;-automata under the probable semantics is 
that all relevant decision problems (checking emptiness, universality or equivalence) are undecidable. 
The undecidability results for PBA have several important consequences. First, the concept of PBA is 
not adequate for solving algorithmic problems that are related to the emptiness or universality problems. 
This, e.g., applies to the verification of nondeterministic systems against PBA-specifications. Second, 
PBA can be viewed as a special instance of partially-observable Markov decision processes (POMDPs) 
which are widely used in various areas, including robotics and stochastic planning (see, e. g., lfT8l[T3l[T0l ) 
and the negative results established for PBA yield the undecidability of various verification problems for 
POMDPs. 

For probabilistic Biichi automata with the almost-sure semantics we obtain a completely different 
picture. They are less powerful and even do not cover the full class of w-regular languages, but still can 
accept languages that are not w-regular. However, the emptiness and universality problem are decidable 
for them. Furthermore, the class of languages that can be accepted by an almost-sure PBA is closed 
under union and intersection, but not under complementation. 

Organization. In Section[2l we briefly recall the definition of nondeterministic cj-automata with Biichi, 
Rabin or Streett acceptance conditions and introduces their probabilistic variants and the probable, 
almost-sure and threshold semantics. The following three sections mainly deal with probabilistic au- 
tomata under the probable semantics. Results on the expressiveness and efficiency of probabilistic Biichi, 
Rabin and Streett automata are summarized in Section[3] Composition operators for PBA under the prob- 
able semantics are addressed in Section 01 Decision problems for PBA and the relation to POMDPs will 
be discussed in Section [5] Section [6] summarizes the main results for the almost-sure and threshold 
semantics. Finally, Section|7] contains some concluding remarks. 

The material of this paper is a summary of the results presented in the papers EH3. Further details 
can be found there and in the thesis by Marcus GroBer [8]. 
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2 Nondeterministic and probabilistic a>automata 

Throughout the paper, we assume some familarity with classical nondeterministic automata over finite 
or infinite words and refer to |[T9l l7l for details. We first recall some basic concepts of nondeterministic 
^-automata and then adapt these concepts to the probabilistic setting. 

Definition 1 Nondeterministic u>automata. 

A nondeterministic w-automaton is a tuple M = S, Qo , Acc) , where 

• Q is a finite nonempty set of states, 

• £ is a finite nonempty input alphabet, 

• 5 : Q x £ — > 2*3 is a transition function, 

• QoQQ is the set of initial states, 

• Acc is an acceptance condition (which will be explained below). 

M is called deterministic if \Qq\ = 1 and \5(q,a) \ = 1 for all q G Q and a£l. 

The intuitive operational behavior of a nondeterministic w-automaton J\f for an infinite input word 
a = a\ 0203 . . . G r w is as follows. The computation starts in a nondeterministically chosen initial 
state qo G Qo- Then, M attempts to read the first letter a\ in state qo. If qo does not have an outgo- 
ing ai-transition (i.e., 5(qo,cn) = 0) then the automaton rejects. Otherwise, the automaton reads the 
first letter a\ and chooses nondeterministically some state q\ G 5(qo,a\). It then attempts to read the 
remaining word 0203 .. . from state q\. That is, the automaton rejects if 5{q\,a,2) = 0. Otherwise the au- 
tomaton reads letter 02 and moves to some state q 2 G 5{q\,a2), and so on. Any maximal state-sequence 
7T = qoQiQ2 •■• that can be obtained in this way is called a run for a. We write inf (it) to denote the set of 
states p G Q that appear infinitely often in n. Each finite run qo q\ . . . qi (where M fails to read letter aj + i 
in the last state qi because 5{qi,ai + \) is empty) is said to be rejecting. The acceptance condition Acc 
imposes a condition on infinite runs and declares which of the infinite runs are accepting. Several accep- 
tance conditions are known for nondeterministic w-automata. We will consider three types of acceptance 
conditions: 

Buchi: A Biichi acceptance condition Acc is a subset F of Q. The elements in F are called final or 
accepting states. An infinite run it = qoq\ q 2 ■ ■ ■ is called (Biichi) accepting if it visits F infinitely 
often, i.e., inf (tt) f\F ^ 0. 

Streett: A Streett acceptance condition Acc is a finite set of pairs {Hi,K{) consisting of subsets H[,Ki 
of Q, i.e., Acc = {(Hi,K\), . . . , (iJ^,iQ)}. An infinite run tt = qoq\q%... is called (Streett) 
accepting if for each I G {1,...,^} we have: inf(7r) n Hi ^ or inf (7r) n K\ = 0. 

Rabin: A Rabin acceptance condition Acc is syntactically the same as a Streett acceptance condition, 
i. e., a finite set Acc = {(H\,K\), . . . , (Ht,K£)} where FL\,K\ C Q for 1 < I < t. An infinite run 
tt = qo Qi 12 ■ ■ ■ is called (Rabin) accepting if there is some I G {1, ... ,£} such that inf (n) D Hi = 
and inf(vr)n^ / 0. 

Using LTL-like notations, a Streett condition can be understood as a strong fairness condition and a 
Rabin condition as its dual. 

/\ (OOK^OOHt) (Streett) 
i<i<e 

V (□0#jA0D-'#i) (Rabin) 
i<i<e 
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Clearly, a Biichi acceptance condition F can be viewed as a special case of a Streett and Rabin condition 
with a single acceptance pair, namely {(F,Q)} for the Streett condition and {(0,F)} for the Rabin 
condition. 

The accepted language of a nondeterministic cj-automaton J\f with the alphabet E, denoted C(N), is 
defined as the set of infinite words a € 1^ that have at least one accepting run in M. 

C(Af) == : there exists an accepting run for a in A/"} 

In what follows, we write NBA to denote a nondeterministic Biichi automaton, NRA for nondeterministic 
Rabin automata and NSA for nondeterministic Streett automata. Similarly, the notations DBA, DRA and 
DSA are used to denote deterministic w-automata with a Biichi, Rabin or Streett acceptance condition. 

It is well-known that the classes of languages that can be accepted by NBA, DRA, NRA, DSA or 
NSA are the same. These languages are often called uo-regular and represented by w-regular expres- 
sions, i. e., finite sums of expressions of the form afi w where a and (5 are ordinary regular expressions 
(representing regular languages over finite words) and the language associated with f3 is nonempty and 
does not contain the empty word. In the sequel, we will identify ^-regular expressions with the induced 
o;-regular language. 

While deterministic ^-automata with Rabin and Streett acceptance (DRA and DSA) cover the full 
class of u;-regular languages, DBA are less powerful as, e. g., the language (a + b)*a w cannot be recog- 
nized by a DBA. Hence, the class of DBA-recognizable languages is a proper subclass of the class of 
u;-regular languages. 

Probabilistic ^-automata can be viewed as nondeterministic w-automata where the transition function 
5 specifies probabilities for the successor states. That is, for any state p and letter a E £ either p does not 
have any a-successor or there is a probability distribution for the a-successors of p. 

Definition 2 Probabilistic u>automata. A probabilistic w-automaton is a tuple V = (<3,£,5,/Uo, Acc), 
where 

• Q is a finite nonempty set of states, 

• Z is a finite nonempty input alphabet, 

• 5 : Q x Z x Q — > [0, 1] is a transition probability function such that for all p G Q and a <G T we 
have: E ? eQ<KP,a,g) G {0,1}, 

• fiQ : Q — > [0, 1] is the initial distribution, i. e., £ gG Q IM)(l) = 1> 

• Acc is an acceptance condition (as for nondeterministic ^-automata). 

We refer to the states qo <G Q where ^o(<7o) > as initial states. If p is a state such that 5(q, a,p) > then 
we say that q has an outgoing a-transition to state p. 

Acceptance conditions can be defined as in the nondeterministic case. In this paper, we just regard 
Biichi, Rabin and Streett acceptance and use the abbreviations PBA, PRA and PSA for probabilistic 
Biichi automata, probabilistic Rabin automata, and probabilistic Streett automata, respectively. 

The intuitive operational behavior of a probabilistic u;-automaton V for a given input word 
a = a\<i2 ■ ■ ■ £ r w is similar to the nondeterministic setting, except that all choices are resolved prob- 
abilistically: the initial state is chosen according to the initial distribution /j,q, and if q is the current state 
and a the next input letter then V moves with probability 5(q,a,i + i,p) to state p. If there is no outgoing 
a-transition from q, i. e., if £ pG q S(q, a,p) = 0, then V rejects. As in the nondeterministic case, the result- 
ing state-sequence ir = popi pi ■ . . G Q* U Q u is called a run for a in V. Acceptance of a run according 
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to a Biichi, Rabin or Streett acceptance condition is defined as in the nondeterministic setting. While ac- 
ceptance of an infinite word in a nondeterministic ^-automata requires the existence of an accepting run, 
a probabilistic ^-automaton accepts an infinite input word a if the probability for the generated sample 
run to be accepting is "sufficiently large". 

Acceptance probability and accepted language. Given an infinite word a G E"\ the acceptance prob- 
ability Pr^ (cr) for a in V denotes the probability measure of the accepting runs for a in V. The formal 
definition of the acceptance probability relies on the view of an input word a G as a scheduler when V 
is treated as a Markov decision process, i.e., an operational model for a probabilistic system where in 
each state q the letters that can be consumed in q are treated as actions that are enabled in q. Given a 
word/scheduler a = a\ 0,2 03 . . . 6 the behavior of V under a is given by a Markov chain M. a where 
the states are pairs (q,i) where q G Q stands for the current state and i is a natural number > 1 that 
denotes the current word position. Stated differently, state (q,i) in the Markov chain A4 a stands for the 
configuration that V might have reached state q after having consumed the first i—l letters a\, . . . ,a£_i 
of the input word a. Assuming that S(q,ai+i, ■) is not the null function, the transition probabilities from 
state (q,i) are given by the distribution S(q, cij+i,-), i. e., from state (q,i) the Markov chain M a moves 
with probability 6(q,a,i+i,p) to state (p,i + 1). In case that S(q,ai + \,-) = then (q,i) is an absorbing 
state, i. e., a state without any outgoing transition. The runs for a in V correspond to the paths in A4 a - 
We can now apply the standard concepts for Markov chains to reason about the probabilities of infinite 
paths and define the acceptance probability for the infinite word a in V, denoted Pr^(a) or briefly Pr(cr), 
as the probability measure of the accepting runs for a in the Markov chain M. a . 

For the definition of the accepted language, we distinguish three semantics for probabilistic u- 
automata. The probable semantics assigns to V the set of infinite words a such that the accepting runs 
for a have positive measure. Under the almost-sure semantics a word a is accepted by V if almost all 
runs for a are accepting. (The formulation "almost all runs have property X" means that the probability 
measure of the runs where property X does not hold is 0.) The threshold semantics relies on a fixed 
threshold A that serves as strict lower bound for the acceptance probability for all accepted words: 



Equivalence of w-automata means that their accepted languages agree. The notion of the size, de- 
noted \V\, of an u-automaton V is used here as follows. The size of a PBA is simply the number of 
states. The size of a probabilistic Rabin or Streett automaton denotes the number of states plus the 
number of acceptance pairs. 

Example 3 Probabilistic Biichi automata (PBA). In the pictures for PBA we attach the probabilities 
5(q,a,p) to the a-labeled edge from q to p, provided that < S(q,a,p) < 1. An a-labeled egde from q 
to p without any probability value indicates that 6(q, a,p) = 1 (in which case p is the unique a-successor 
of q). Similarly, the initial distribution is depicted by attaching the value fio(q) to an arrow pointing to q, 
provided that q is an initial state and fio(q) < 1. For PBA, we depict the accepting states (i. e., the states 
q € F) by squares, non-accepting states by circles. The PBA V over the alphabet £ = {a, b} shown in 
the left part of Figure[T]has a single initial state qo. Its Biichi condition is given by F = {q{\. 

Let us first observe that each word a that is accepted by V must be contained in the language 
{a-\-b)*a w of the NBA that results from V by ignoring the probabilities. Indeed all words with only 



C>°(V) = {a€l< 

c=\v) = {aer 

C> X (V) = {a€l< 




(a) > 0} 

(*) = !} 
(a) > A} 
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PBA V PBA V' 

Figure 1 : Examples for PBA V (left) and V' (right) 



finitely many 6's are accepted by V under the probable semantics, i. e., we have C (V) = (a + 6)*a w , 
while the almost-sure semantics yields C =l (V) = b*a u . To see why, let us compute the acceptance prob- 
ability for the words a G (a + b)*a u . Clearly, if a = a? then Pr 7> (cr) = 1. Suppose now that a contains 
at least one b and let k be the total number of a's that appear before the last b in a, i. e., if a = c\ . . . C(ba u 
then k = \{i G {1, .. . ,£} : Ci = a}\. With probability 2~ k , the current state is qo after reading ci ... eg. But 
then V can read b and will almost surely move to q\ when reading the suffix a^. Thus, Pr p (a) = 2~ k 
which yields that a G C =l (V) iff k = (i. e., if <r € 6*0") and that all words in (a + 6)*acj belong to 
£ >0 (V). 

Regard the PBA V' over the alphabet £ = {a,b,c} shown in the right part of Figure Q] Let us first 
observe that the underlying nondeterministic Biichi automaton (NBA) that we obtain by ignoring the 
probabilities has an accepting run for each infinite word in (ab + ac)^ with infinitely many 6's, no matter 
whether there are only finitely c's or infinitely many c's. Thus, the accepted language of the NBA is 
Uac)*ab) . This language is different from the accepted language of the PBA V' under the probable 
and almost-sure semantics: 

£>°(p>) = (ab + ac)*(aby, L=\V) = {abf . 

Clearly, all accepted words a G C >0 (V) belong to ((ac)*a&) u; . Any word a in (ab + ac) w with infinitely 
many c's is rejected by V' as almost all runs for a are finite and end in state p\, where the next input 
symbol is c and cannot be consumed in state p\. Thus, £ >0 (V') C (ab+ac)*(ab) w . Given an input word 
a G (ab + ac)* (ab) w ', say a = x(ab) UJ where x G (ab + ac)*, then with positive probability V' generates 
the run fragment PQP2P0P2 ■ --P0P2P0 when reading x. For the remaining suffix (a6) w , V' can always 
consume the next letter and almost surely V' will visit p\ and P2 infinitely often. This yields Pr v (a) > 
and a G C >0 {P'). 

Clearly, we have C =1 (V') C £ >0 (V'). Using an argument as above, it is clear that no word in 
C =l (V') contains letter c. The runs for the word (ab) w will almost surely visit state pi infinitely often. 
This yields C=\V) = (ab) u . 

The precise acceptance probability for a G {a,b,c} w is as follows. If a ^ (ab + ac)* (ab) w then 
Pr (er) =0. If a G (ab + ac)*{ab) w and letter c appears k times in a then Pr v '(a) = 2- k . Thus, 
e.g., for threshold 0.1, the accepted language C >0A (V) consists of all words a G (ab + ac)* (ab)^ that 
contain three or fewer c's. o 
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3 Expressiveness and efficiency of PBA 

In the following three sections, we put the focus on probabilistic ^-automata with the probable seman- 
tics. Results for the almost-sure and threshold semantics are summarized in Section [6] Unless stated 
differently, we simply say PBA to denote a PBA with the probable semantics. 

We start with a discussion on the expressiveness and efficiency of PBA compared to their nondeter- 
ministic counterparts. At the end of this section, we will show that as in the nondeterministic case, Buchi 
acceptance is as powerful as Streett and Rabin acceptance. 

PBA and a>regular languages. DBA can be viewed as special instances of PBA (we just have to as- 
sign probability 1 to all edges in the DBA and deal with the initial distribution that assigns probability 1 to 
the unique initial state). As the language (a + b)*a w is recognizable by a PBA with the probable seman- 
tics (see Example [3]), PBA are strictly more expressive than DBA, i. e., the class of DBA-recognizable 
languages is a proper subclass of the class of languages £ > (V) for some PBA V. Indeed all w-regular 
languages can be represented by a PBA with the probable semantics: 

Lemma 4 From NBA to PBA under the probable semantics. For each NBA J\f there exists a PBA V 
such that £ >0 (V) = C{M). 

Proof: A transformation from NBA into an equivalent PBA is obtained by using NBA that are deter- 
ministic-in-limit. These are NBA such that 5(q,a,p) G {0, 1} for all states p and q that are reachable 
from some accepting state and all letters a € £. That is, as soon as an accepting state has been reached 
the behavior from then on is deterministic. Courcoubetis and Yannakakis [6] presented some kind of 
powerset construction which turns a given NBA J\f into an equivalent NBA A/"det that is deterministic- 
in-limit. If we now resolve the nondeterministic choices in A/"d e t by uniform distribution^ then A/"det 
becomes a PBA that accepts the same language as J\f (and Mdet)- D 

We now address the question whether each PBA can be transformed into an equivalent NBA. Sur- 
prisingly, this is not the case, as there are PBA where the accepted language is not u-regular. An example 
for a PBA V = V\ where the acepted language under the probable semantics is not w-regular is given in 
Figure El 




where A G]0,1[ 



a 



Figure 2: PBA V\ accepts a non-cj-regular language 

Here, A is an arbitrary real number in the open interval ]0, 1 [. 

Lemma 5. The language of the PBA V\ under the probable semantics is not NBA-recognizable, i. e., 
C >0 (V) is not uj-regular. 

If q is a state in A/^jet an d a g Z such that q has k a-successors q\, . . . ,q^ then we define S(q, a, qi) = j for 1 < i < k and 
5(q, a,p) = for all states p ^ {qi ,.-.,%}. Similarly, if Qq is the set of initial states in A/"d e t and Qo is nonempty then we deal 
with the initial distribution no that assigns probability 1/jQoj to each state in Qo- 
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Proof: The PBA V\ accepts the language 

oo 

C >0 (V X ) = {a k> ba k2 ba k3 b... :Yl(l-X ki ) > }. 

i=i 

The convergence condition which requires the infinite product over the values 1 — X kt to be positive can 
easily be shown to be non-w-regular, i. e., £ >0 (V\) cannot be recognized by an NBA. 

To see that, indeed, C >0 (V X ) agrees with the above language, let us first observe that all words in 
C >0 (V\) must contain infinitely many b's. Note that if a input word a ends with the suffix a w then almost 
all infinite runs for a will eventually enter state q\ and stay there forever. As V\ cannot consume two 
consecutive 6's, all words in £ >Q (P\) have the form a k[ ba kl ba k3 b. . . where k\,k2,--- is a sequence of 
positive natural numbers. We now show that 

oo 

Pr v *(a kl ba k2 ba k3 b...) = YK 1 -^)- 

i=l 

The factors 1 — X ki stand for the probability to move from state qo to q\ when reading the subword 
a ki . With the remaining probability X kl , the automaton V\ stays in state qo, but then letter b at position 

fci H + k{ + i of the input word a kl ba k2 ba k} b . . . cannot be consumed and V\ rejects. Hence, the 

probability for run fragments of the form qo... qoqi ...q\qo that are generated while reading the subword 
a ki b is precisely 1 — X ki . This yields that the infinite product of these values agrees with the acceptance 
probability for the input word a k[ ba kl ba k3 b. ... □ 

As a consequence of Lemma|4]and Lemma[5]we get that PBA with the probable semantics are more 
powerful than NBA. This result should be contrasted to the case of finite automaton where the probable 
semantics turns PFA into ordinary NFA, and thus, PFA with the probable semantics represent exactly the 
class of u;-regular languages. 

Corollary 6. The class of languages that are accepted by a PBA strictly subsumes the class ofu-regular 
languages. 

The PBA V\ can also serve to illustrate that the probable semantics is sensititive to modifications of 
the transition probabilties. Consider two values A and v g]0, 1[ with A < v. For any sequence (fcj)j>i 
of natural numbers k\ where the infinite product over the values 1 — u ki converges to some positive 
value, also the infinite product over the values 1 — X ki is positive, as we have \ — v ki < 1 — X ki . Thus, 
£ >0 (Pu) Q C >0 (P\). However, whenever A < v then C >0 {V V ) is a proper sublanguage C >Q (P\) as 
there are sequences (&i)j>i such that the product of the values 1 — X ki converges to some positive real 
number, while the product of the values 1 — v ki has value (2J. Hence: 

Lemma 7. IfX < v then C >0 (V V ) ± £ >0 (^a)- 

Thus, the languages of PBA are sensitive to the distributions for the successor states. That is, if 
we are given a PBA and modify the nonzero transition probabilities then also the accepted language 
might change. This property is surprising since the definition of the accepted language just relies on a 
qualitative criterion: the acceptance probability must be positive, but might be arbitrarily small. This 
should be opposed to the verification of finite-state Markov decision processes where it is known that 
whether or not a given linear time property holds with positive probability just depends on the underlying 
graph, but not on the concrete transition probabilities. 
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Efficiency. The transformation from NBA into an equivalent PBA with the probable semantics de- 
scribed in the proof of Lemma[4]relies on a certain powerset construction that turns a given NBA N into 
an NBA that is deterministic-in-limit and can be interpreted as a PBA. This transformation can cause an 
exponential blow-up. In fact, in the worst-case, the exponential blow-up cannot be avoided for the trans- 
formation from NBA to PBA as there are families (L n ) n >\ of w-regular languages that are accepted by 
NBA of linear size, while each PBA for L n has Q.(2 n ) states. An example for such a family of languages 
is 

L n = {{a + bfa{a + b) n cy. 

Language L n is recognizable by an NBA with n+1 states which guesses nondeterministically for any 
word position i where the input word contains an a whether letter c will appear at word position i+n. 
Since there is no upper bound on the distance between the word positions of the c's in the words in L n , 
any PBA for L n needs to store the positions of letter a among the last n letters (see Q). Hence, the size 
of any PBA for L n is exponential. Vice versa, there are also examples for w-regular languages where 
probabilism allows for a more compact representation than nondeterminism. Let 

L 'n = { x lt ■ x ,y€ {a,b}*,\y\ = n}. 

Lemma 8. Each NSA for L' n has 2 n /n or more states in each NSA for L' n , while there exist PBA V n 
consisting of 0(n) states with £ >0 (V n ) = L' n . 

Proof: The lower bound 2 n /n for the number of states in any NSA for L' n is obtained by verifying that 
given two words y=c\C2 ...c n and z = d\dq. ■ ■ ■ d n of length n such that 

d\d 2 ...d n (£{ CiQ+i . . . c n ci . . . Ci-i : 1 <i<n} 

then the "accepting cycles" for the words y^, z w G L' n do not intersect. 

It remains to show the existence of PBA of linear size for L' n . Let V n be the following PBA. The 
states of V n are l a , . . . ,n a , lb, . . . ,rift. Thus, V n has 2n states. States \ a and 1^ are initial, both have 
probability 0.5 under the initial distribution. All states are accepting. (Thus, any infinite run in V n is 
accepting.) V n has the following transitions. From any state k a with 1 < k < n there is an a-transition 
to state (k+l) a and a 6-transition to state (fe+l)f,. All these transitions have probability 0.5. All states, 
except for state rib, have an a-transition to state \ a . These transitions have probability 0.5, except for 
the transition from n a to l a which has probability 1. Similarly, from any state fc& with 1 < k < n there 
is an a-transition and a 6-transition to state (A;+l)& with probability 0.5. All states, except for state n a , 
have a 6-transition to state 1& with probability 0.5 except for state rib which has a 6-transition to lb with 
probability 1. 

The idea of this construction is as follows. While scanning an infinite input word 

<7 = cic 2 c 3 ... G {a,6} w , 

V n chooses at random word positions i by moving to state l a (if c, = a) or state 1& (if q = 6) and checks 
whether c n+ i = C{ via following the path 

K ► 2 Cl ► ► n Ci 

and rejecting (if c n+ i ^ cf) or returning to state l Ci (if c n+ j = Cj) and choosing the next word position j, 
and so on. If a £ L' n then there are infinitely many word positions i such that c n+ i ^ c, and almost 
surely V n will pick such a word position and reject in state n Ci . If a = c\oiOi . . . G L' n then there exists 
some index I such that q = c n +i for all i>£. After reading the ^-th letter, V n will be in state l C( with 
probability > 2~ e . From then on, V n will never reject and the resulting runs are accepting. Hence, 
Pr p "(o-)>0. □ 
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Streett and Rabin acceptance. The three types of probabilistic u-automata (Biichi, Rabin, Streett) are 
equally expressive. As the Biichi acceptance condition can be rewritten as a Rabin or Streett acceptance 
condition, each PBA can be viewed as a PRA or as a PSA with the same accepted language. But we 
can establish a stronger result stating that each PBA can be transformed into a 0/1 -PRA which means a 
PRA Vr such that for each word a, the acceptance probability for a is either or 1. This result can be 
viewed as the probabilistic analogue to the well-known fact that each NBA can be transformed into an 
equivalent deterministic Rabin automaton. The idea for this transformation is to design a 0/1 -PRA Vr 
that generates up to n sample runs of V and checks whether at least one of them is accepting, where n is 
the number of states in V. If so then Vr accepts, otherwise it rejects. For the details of this construction 
we refer to EHH. 

Theorem 9 From PBA to 0/1-PRA. For each PBA V there exists a 0/1-PRA Vr such that 

£>°(V) = £ >0 (Vr). 
Vice versa, there are polynomial transformations from PRA and PSA to PBA: 
Theorem 10 Polynomial transformations from PBA to PRA and PSA. 

(a) Given a PRA Vr with £ acceptance pairs there exists a PBA V of size 0(£\Vr\) such that 
C>°(V) = £ >0 (V R ). 

(b) Given a PSA Vs with £ acceptance pairs there exists a PBA of size 0(£ 2 \Vs\) such that 
C>°(V)=C >0 {Vs). 

The transformation from PRA to PBA is roughly the same as in the nondeterministic case. The con- 
struction of a PBA of size 0(£ 2 \Vs\) from a given PSA Vs, however, crucially relies on the probabilistic 
semantics. In fact, it is worth noting that in the nonprobabilistic case it is known (see ifTTIO that there 
are families (L n ) n >o of languages L n C £ w that are recognizable by nondeterministic Streett automata 
of size 0(n), while each nondeterministic Biichi automaton for L n has 2 n or more states. Thus, the 
polynomial transformation from Streett to Biichi acceptance is specific for the probabilistic case. 

4 Composition operators for PBA 

The most important composition operators for any class of languages over infinite words are the standard 
set operations union, intersection and complementation. In fact, the class of PBA-recognizable languages 
is closed under all three operations. 

Theorem 11. The class of languages C >0 (V) for some PBA V is closed under union, intersection and 
complementation. 

Given two PBA V\ and V2 over the same alphabet with initial distributions n\ and respectively, 
then a PBA V for the language C >0 (Vi ) U £ >0 (V 2 ) can be obtained by the disjoint union of V\ and Vi_ 
with the initial distribution fi(q) = \[ii{q) if q is a state in Vi. If Fi and F2 are the sets of accepting states 
in V\ and V2, respectively, then V requires to visit F\ VJF2 infinitely often. 

An operator for PBA with the probable semantics that realizes intersection can be designed by reusing 
ideas that are known for NBA. Given two PBA V\ and V2 over the same alphabet, we use a product 
construction V\ x V2 (which runs V\ and V2 in parallel) and equip V\ x V 2 with a Streett acceptance 
condition consisting of two acceptance pairs. One of the acceptance pairs requires that an accepting 
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state of V\ is visited infinitely often, the other one stands for the acceptance condition of V%. This PSA 
V\ x V% can then be transformed into an equivalent PBA (part (b) of Theorem [TOl). 

The most interesting operator is complementation. Given a PBA V with L = C >0 (V) C E w , the idea 
for the construction of a PBA V for the language L = £ w \ L is somehow similar to the complementation 
of NBA via Safra's determinisation operator [16 ] and relies on the transformations sketched in Figure [3] 
In the first step we apply the transformation mentioned in Theorem |9l while the last step relies on part 

PBA V with 0/1-PRA 7?^ 0/1 -PSA Vs PBA V with 

L = C >0 (V) fori fori C >0 (V) = L 



Figure 3: Complementation of a PBA 



(b) of Theorem [TOl Recall that a 0/1-PRA denotes a PRA Vr where the acceptance probabilities for 
all words are or 1, i. e., ?r VR {a) G {0, 1} for each word a G L u . Thus, C >0 {Vr) = C= l {Vr) and for 
transforming the 0/1-PRA Vr into a 0/1-PSA Vs for the complement of £ >0 (Vr) we may simply use 
the duality of Rabin and Streett acceptance. That is, syntactically Vr and Vs agree (but Vs is viewed 
as a Streett and Vr as a Rabin automaton). The size of the resulting PBA V for L can be exponentially 
larger than the size of V due to the powerset construction used in the generation of a 0/1-PRA. 



5 Decision problems for PBA 

For many applications of automata-like models, it is important to have (efficient) decision algorithms for 
some fundamental problems, like checking emptiness or language inclusion. For instance, the automata- 
based approach ll20l for verifying w-regular properties of a nondeterministic finite-state system relies on 
a reduction to the emptiness problem for NBA. Unfortunately, the emptiness problem and various other 
classical decision problems for automata cannot be solved algorifhmically for PBA: 

Theorem 12 Undecidability of PBA. The following problems are undecidable: 

• emptiness: given a PBA V, does £ >0 (V) = hold? 

• universality: given a PBA V with the alphabet £, does £ >0 (V) = hold? 

• equivalence: given two PBA V\ and Vi, does C >0 {V\) = £ >0 (V2) hold? 

To prove undecidability of the emptiness problem, we provided in [2] a reduction from a variant of the 
emptiness problem for probabilistic finite automata (PFA) which has been shown to be undecidable ifTTt . 
Undecidability of the universality problem then follows by the effectiveness of complementation for 
PBA. Undecidability of the PBA-equivalence problem is an immediate consequence of the undecidability 
of the emptiness problem (just consider V\=V and V2 a PBA for the empty language). 

A consequence of Theorem [l2]is that PBA are not appropriate for verification algorithms. Consider, 
e. g., a finite-state transition system T and suppose that a linear-time property p to be verified for T 
is specified by a PBA V in the sense that £ >0 (V) represents all infinite behaviors where property p 
holds. (Typically p is a language over some alphabet Z = 2 AP where AP is a set of atomic propositions 
and the states in T are labeled with subsets of AP.) Then, the question whether all traces of T have 
property p is reducible to the universality problem for PBA and therefore undecidable. Similarly, the 
question whether T has at least one trace where p holds is reducible to the emptiness problem for PBA 
and therefore undecidable too. 



14 



Probabilistic Automata over Infinite Words 



Another important consequence of Theorem [T2]is that it yields the undecidability of the verification 
problem for partially observable Markov decision processes (POMDPs) against w-regular properties. 
POMDPs provide an operational model for stochastic systems with non-observable behaviors. They 
play a central role in many application areas such as mobile robot navigation, probabilistic planning 
task, elevator control, and so on. See, e. g., lTT8l [T2l [T3l [Toll . The syntax of a POMDP can be defined as 
for probabilistic w-automata, except that the acceptance condition has to be replaced with an equivalence 
relation ~ on the states which formalizes which states cannot be distinguished from outside. The ele- 
ments in the alphabet £ are viewed as action names. The goal is then to design a scheduler 5 that chooses 
the actions for the current state and ensures that a certain condition X holds when the choices between 
different enabled actions in the POMDP M. are resolved by 5. For his choice the scheduler may use the 
sequence of equivalence classes that have been passed to reach the equivalence class of the current state. 
That is, the scheduler is supposed to observe the equivalence classes, but not the specific states. (Such 
schedulers are sometimes called "partial-information schedulers" or "observation-based schedulers".) 

The emptiness problem for PBA is a special instance for the scheduler-synthesis problem for 
POMDPs. Given a PBA V = (Q,L,S,fM,,F), we regard the POMDP M = (Q,L,6,uq,~) where ~ 
identifies all states and ask for the existence of a scheduler that ensures that F will be visited infinitely 
often with positive probability. We first observe that the infinite words over £ can be viewed as sched- 
ulers for M., and vice versa. Hence, £ >0 (V) is nonempty if and only if there is a scheduler 5 such that 
Prj'fpOF) > 0, where Prg^DOF) denotes the probability that M visits F infinitely often when 5 
is used to schedule the actions in M.. Similarly, the universality problem for PBA can be viewed as a 
special instance of the problem where we are given a POMDP hA and a set F of states and ask for the 
existence of a scheduler 5 such that Pr^(()\JiF) = 1 where Pr^^HF) denotes the probability that M. 
under scheduler 5 eventually enters F and never leaves F from this moment on. Thus: 

Theorem 13 Undecidability results for POMDPs. 

The following problems are undecidable: 

• given a POMDP M and a set F of states, decide whether 35. Pr^ 1 (OOF) > 0, 

• given a POMDP M and a set F of states, decide whether 35. Pr^(()UF) = 1. 

The result of Theorem [13] is remarkable since the corresponding questions for fully observable 
Markov decision processes (i. e., POMDPs where the ~-equivalence classes are singletons) are decidable 
in polynomial time. 

6 The almost-sure and threshold semantics 

So far, we concentrated on the probable semantics of probabilistic u>automata. We will briefly summa- 
rize the main results on the almost-sure and threshold semantics. 

PBA with the almost-sure semantics are less expressive than PBA with the probable semantics. They 
even do not cover the full class of u>regular languages. For instance, the u;-regular language (a + 6)*a w 
cannot be recognized by a PBA with the almost-sure semantics. Since the complement (a*b) w of this 
language is recognizable by a deterministic Biichi automaton (and therefore also by a PBA with the 
almost-sure semantics), PBA with the almost-sure semantics are not closed under complementation. Fur- 
thermore, there are PBA where the almost-sure semantics yields a non-u-regular language. An example 
is the language 
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which can be shown to be recognizable by a PBA with the almost-sure semantics. However, the class of 
languages C =1 (V) for some PBA V is closed under union and intersection. For PBA with the almost- 
sure semantics, the emptiness and universality problem are decidable. Indeed one can even show that 
given a POMDP M. and a set F of states in M. then the questions 

does there exists a scheduler S such that Prj^^QF) = 1? 
does there exists a scheduler S such that Pr^(()DF) > 0? 

are decidable by a certain powerset construction. Using the above mentioned fact that PBA can be viewed 
as special instances of POMDPs, one obtains the decidability of the emptiness and universality problem 
for PBA with the almost-sure semantics. 

It should be noticed that the above results on the almost-sure semantics are specific for the Biichi 
acceptance condition. For Rabin or Streett acceptance, the almost-sure semantics is as expressive as the 
probable semantics. This is a consequence of Theorems [9] and [10] which show that PRA with the almost- 
sure semantics are as expressive as PRA (and PBA) with the probable semantics. Thus, the emptiness, 
universality and equivalence problems for PRA with the almost-sure semantics are undecidable. 

The threshold semantics is more powerful than the probable semantics. Indeed for each PBA V and 
threshold A there exists a PBA V such that £ >0 (V) = C >X (V). Furthermore, there are transformations 
to stretch and relax acceptance probabilities which yields that whenever A, v e]0, 1 [ and V is a PBA then 
there exists a PBA V' such that £ >X (P) = £ >U (P'). That is, all thresholds define the same class of 
languages. Using known results on the expressiveness of probabilistic finite automata (PFA) lfl5l [I4l . 
one can show that there are threshold languages £ >X (T) that cannot be recognized by PBA with the 
probable semantics. The undecidability of all relevant algorithmic problems for PBA with the threshold 
semantics is clear from the undecidability of corresponding problems for PFA iTTTTl . As far as we know, 
closure properties under composition operators have not yet been studied for PBA with the threshold 
semantics. 

7 Conclusion 

We gave a summary of the fundamental properties of probabilistic acceptors for infinite words formal- 
ized by probabilistic w-automata with Biichi, Rabin or Streett acceptance conditions. The results show 
some major differences to nondeterministic (or alternating) oj-automata concerning the expressiveness, 
efficiency and decidability. 

Beside being of theoretical interest, we believe that PBA could be useful in several application areas. 
We briefly sketched the connection between probabilistic u;-automata and POMDPs. Since PBA arise 
as special instance of POMDPs all negative results for PBA (undecidability) carry over from PBA to 
POMDP. Vice versa, it seems that for many algorithmic problems for POMDPs, algorithmic solutions 
for probabilistic u;-automata (e. g., PBA with the almost-sure semantics) can be combined with standard 
algorithms for (fully observable) Markov decision processes to obtain an algorithm that solves the anal- 
ogous problem for POMDPs. Another application of probabilistic u;-automata is run-time verification 
where special types of PBA can serve as probabilistic monitors 0]. Given the wide range of appli- 
cation areas of probabilistic finite automata, there might be various other applications of probabilistic 
iij-automata. For instance, the concept of probabilistic u;-automata is also related to partial-information 
games with w-regular winning objectives [5] or could serve as starting point for studying quantum au- 
tomata over infinite inputs, in the same way as PFA yield the basis for the definition of quantum finite 
automata J9j[l]. 
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